8

Introduction

FIGURE 1.5

BinaryDenseNet.

of parameters equal for a given BinaryDenseNet, they halve the growth rate and double the

number of blocks simultaneously. The architecture of BinaryDenseNet is shown in Fig. 1.5

MeliusNet [10] presents a new architecture alternating with a DenseBlock, which in-

creases the feature capacity. They also propose an ImprovementBlock, which increases the

quality of the features. With this method, 1-bit CNNs can match the accuracy of the popular

compact network MobileNet-v1 in terms of model size, number of operations, and accuracy.

The building blocks of MeliusNet are shown in Fig. 1.6.

Group-Net [303] also improves the performance of 1-bit CNNs through structural design.

Inspired by a fixed number of binary digits representing a floating point number in a com-

puter, Group-Net proposes decomposing a network into binary structures while preserving

its representability rather than directly doing the quantization via ”value decomposition.”

Bulat et al. [25] are the first to study the effect of neural network binarization on lo-

calization tasks, such as human pose estimation and face alignment. They propose a novel

hierarchical, parallel, and multiscale residual architecture that significantly improves per-

formance over the standard bottleneck block while maintaining the number of parameters,

thus bridging the gap between the original network and its binarized counterpart. The new

architecture increases the size of the receptive field, as well as the gradient flow.

LightNN [57] replaces multiplications with one shift or a constrained number of shifts

and adds, which forms a new kind of model. The experiments show that LightNN has better

accuracy than BNNs, with only a slight increase in energy.

FIGURE 1.6

Building blocks of MeliusNet (c denotes the number of channels in the feature map).